For two dimensional (2D) displays, captioning (e.g., subtitling, closed-captioning, and the like) can be arbitrarily rendered into a pre-existing 2D image by a video playback device, or by compositing software. The inserted 2D captioning will appear by occlusion to be in front or behind features in the pre-existing image. Auxiliary data provided with the captioning can specify simple 2D placement locations and styles. For example, a caption's auxiliary data from a broadcast can specify either roll-up, pop-up or paint-on captioning. Roll-up captioning typically appears at the bottom of the screen, while pop-up captioning appears anywhere on the screen. Paint-on captioning uses a predetermined stationary block on the screen.
However, these conventional techniques are unsuitable for a pre-existing 3D image. Arbitrary rendering of captioning in 3D space results in captioning existing in the background (e.g., behind a pre-existing feature) but rendered in the foreground (e.g., in front of the pre-existing feature), or oppositely existing in the foreground and rendered in the background. In other words, the captioning can appear by occlusion to be in front of a feature while appearing by interocular disparity to be behind the same feature. This conflict confuses the human visual system which breaks the illusion of depth in the 3D image, as well as cause eye fatigue for a viewer.
These pitfalls can be avoided with manual eyes-on analysis of each 3D captioning insertion during post-production. That is to say, a human editor confirms the appropriateness of each insertion. As one can imagine, this process is time consuming, costly, and prone to human error. Manual eyes-on analysis is not practical for pre-recorded content and all but impossible for a live broadcast.
Furthermore, manual eyes-on analysis is performed for a specific display device dimension and often yields an unanticipated result for a different display device. Perceptual depth is non-linear with pixel disparity, and pixel disparity is a function of both pixel separation offset and each display device's pixel resolution. Accordingly, inserted captioning can appear in an undesirable location even using the same pixel separation offset. For example, captioning can appear on the different display device to exist beyond infinity causing divergence of the viewer's eyes. The captioning appears beyond infinity when its pixel disparity between right and left eye images is greater than the viewer's interocular separation, which is generally about 65 millimeters for an adult. An unanticipated result can also include illegal positioning of the captioning, such as behind the viewer, which is impossible to achieve on a display.
From the above, it is seen that 3D captioning, as well as 3D object insertion in general, positioned automatically and/or independently of a display device can provide many benefits over conventional 2D and 3D techniques.